Typically, diagnosing the cause of trend shifts in performance data associated with mechanical systems, electrical systems, and electro-mechanical systems involves inconsistent, inefficient manual processes. Engineers and the like who use such diagnostic systems have to choose between data-driven systems and rule-based systems. Data-driven systems, also referred to herein as “case-based systems” or “experience-based systems,” require the incorporation of a relatively large number of examples or validation cases before a given diagnostic system “learns” how to make an accurate diagnosis. Such diagnostic systems are prone to over-fitting data and making important decisions based on infrequent and/or irrelevant information. These diagnostic systems, however, are useful for diagnosing problems where examples or validation cases are plentiful and there is relatively little domain knowledge. In the aircraft engine domain, however, there is a relatively large amount of information related to how an aircraft engine works and why problems occur. Examples or validation cases, however, are not plentiful. In such a domain, experts typically prefer to write rules explaining what they hope to find and how to make diagnoses. These manually written rules suffer from the fact that they do not always match the examples or validation cases perfectly. Differences in the way symptoms are measured and the inability to predict the magnitude and/or speed of symptoms cause the rules to be imprecise, even if they are relatively easily interpreted and corrected by engineers or the like performing manual diagnoses. An automated diagnostic system performing such diagnoses, such as a computerized diagnostic system, has a relatively difficult time correcting the rules in real time.
Additionally, when multiple performance parameters are examined over time, rule-based systems, also referred to herein as “model-based systems,” suffer from model uncertainty (related to the inability to determine how large of a trend shift to correlate to a given problem) and measurement uncertainty (related to the inability to determine the extent of the effect of noise on a given trend shift). Multiple performance parameters must, however, be considered in order to make an accurate diagnosis. Typically, these problems have been addressed via thresholding and the use of trend shift alerts. These trend shift alerts often utilize dimensionality that is too low to make an accurate diagnosis and, historically, rules are only corrected when they fail, i.e., they are not optimized.
Thus, what is needed are consistent, efficient systems and methods that allow for the analysis of performance data and the measurement of trends and trend shifts related to mechanical systems, electrical systems, and electro-mechanical systems. These systems and methods should allow the trend shifts to be compared to models built by experts and diagnose the cause of the trend shifts. What is also needed are systems and methods that allow engineers or the like to enter examples or validation cases against which the models may be evaluated and optimized. This would allow the engineers or the like to verify that the optimizations are appropriate and not over-fit.